Video streaming is an important application over the Internet. The number of available streams, servers and users is rapidly growing and is the leading consumer of bandwidth. The following text provides a short description about how streaming is performed today and one of the problem this introduces.
“Classical” Streaming
Streaming in its original form was performed by transmitting a stream from the server to a client with little or no feedback. If feedback was provided, this occurred in cases the client did not receive data within a given time. The feedback could be done by use or lack of Acknowledgement (ACK) messages if the stream was sent over the TCP protocol or by resending requests for the stream by the client. In many cases the protocol used was RTSP.
Adaptive Bit Rate with “Classical” Streaming
Since the server and the client do not have clear visibility of the medium between them, it is practically impossible for the server to know if a stream with one specific bit-rate could reach the client or whether bandwidth congestion on the way would block the client from receiving it. Therefore, servers adopted different adaptive bit rate methods. In all cases, the servers maintain a few versions of the stream, each one encoded in a different bit-rate. In the general case, if a segment of a stream with a specific bit rate did not reach the client, the server would try to retransmit the stream with a lower bit rate.
Lower bit rates can be achieved by lowering spatial resolution, number of frames per second or simply applying stronger compression to the stream. In cases where the clients received all the transmitted segments, the server could revert to higher bit rate versions of the stream and transmit those. In these cases, the device which decided which bit rates to use was the server.
HTTP Streaming
for different reasons including firewall restrictions and compatibility between Different servers and clients, the market has generally reverted (or is in the process of Reverting) to HTTP streaming. When performing HTTP streaming the client requests a file download over HTTP. In this case, the client may request only parts of a file or all of it. If the client desires, it may stop the downloading of the file and ask the server to download a different file. To sum this up, in the simplest sense, HTTP streaming involves a file download from the server to the client. This type of streaming is often known as Progressive Download (PD).
Adaptive Bit Rate (ABR) with HTTP Streaming
Adaptive bit rate with HTTP streaming is a relatively new technology. When it is used, the server first saves a few different versions of the video clip. The difference is usually a bitrate difference but there might also be resolution or other differences between the versions. The different versions are chopped into small segments usually known as fragments or chunks (these words are used interchangeably in this application) which have a playback duration of a few seconds. The server sends the client a manifest file. The manifest file informs the client which versions are available, the duration of the fragments and where the client can find the different fragments. The client then asks the server to transmit specific fragments of the stream
ABR HTTP streaming can also be used with live streaming. In this case, the manifest file is constantly updated and sent to the client. In some ABR cases, each bit rate chunk is stored as a single file. In other cases, each file holds the entire stream with one specific bit rate and different chunks are simply different segments of the file. Other implementations may include only one file which is segmented by bit rates and chunks.
As in classical streaming, the client and server do not have any information regarding the medium between them. Therefore, a mechanism is needed here also to determine the stream's bit rate. In this case it is the client which makes the decision. Algorithms implemented in the client decide which bit rate chunk to request from the server. The decision may be influenced by several factors such as available buffer space, processing power of the client and bandwidth considerations such as: if the previously requested chunk was not received (probably because of network congestion)—request a lower bit rate fragment next time. Occasionally the client may request a chunk with a bit rate which is higher than the last received chunk.
In all the cases described above, when a stream is encoded for a few versions, it is always encoded in constant bitrate (CBR). Using variable bitrate (VBR) encoding would complicate the mechanisms which try to assess the available bandwidth and would also complicate the actual delivery mechanisms. The chunks, or fragments as they are usually called, have durations of a few seconds. Typical durations range from 2 to 8 seconds. This reduces significantly the granularity and resolution of bitrate decisions. In some cases a finer granularity is needed. These cases may include situations were some streams share the same delivery medium.
Another problem that arises from using CBR encoding is that the visual quality tends to vary. For example, two scenes with different encoding complexity would need to be encoded at two different bitrates in order to achieve similar visual quality. It is obvious therefore, that if all scenes are encoded with the same bitrate, the visual quality would vary. This may be important in a case where two or more clients share the same pipe for streaming. An operator may prefer to allow the devices to receive the stream at a constant video quality, making the viewing experience more acceptable. In order to achieve this, some sort of multiplexing of the streams is needed which would include transforming CBR streams into VBR streams.
Previous Solutions
In non-streaming environments there are some solutions to this problem. These solutions include open-loop and closed-loop statistical multiplexing with partial or full re-encoding. These re-encoding techniques include re-quantization, re-quantization with motion drift correction or full decoding and encoding. In streaming environments there are no known solutions for creating multiplexes of VBR streams. There are, however, transcoders that re-encode the streams and may turn them into VBR streams where the variability of the bitrate depends on the assessed available bandwidth on the pipe. This process is expensive in that it requires significant resources to perform the transcoding. Additionally, each new generation of encoding adds noise to the original content and degrades is quality regardless of the degradation which may occur due to the bitrate reduction.